The Recovery System for Hadoop Cluster
نویسندگان
چکیده
Due to brisk growth of data volume in many organizations, large-scale data processing became a demanding topic for industry as well as for academic fields. Hadoop is widely adopted in Cloud Computing environment for unstructured data. Hadoop is an open source, a java based distributed computing framework, and supports large-scale distributed data processing. In the recent years, Hadoop Distributed File System (HDFS) is popular for huge data sets and streams of operation on it. Availability of Hadoop is the important factor in Cloud Computing. But, in HDFS, Namenode failure affects the performance of the Hadoop cluster. It can be a single point failure. In this paper, we analysed the behaviour of Namenode, what are effects of Namenode failure. This paper presents a scenario to overcome this failure. Our scenario replicates the Namenode on the other Datanode so that the availability of the metadata is increases which will reduce the loss of data as well as delay. Keywords— Hadoop; Cloud Computing; HDFS; Namenode; availability; failure.
منابع مشابه
Adaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملSustainability of Hadoop Clusters
Hadoop is a set of utilities and frameworks for the development and storage of distributed applications in cloud computing, the core component of which is the Hadoop Distributed File System (HDFS). NameNode is a key element of its architecture, and also its “single point of failure”. To address this issue, we propose a replication mechanism that will protect the NameNode data in case of failure...
متن کاملA Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster
Erasure codes, such as Reed-Solomon (RS) codes, are being increasingly employed in data centers to combat the cost of reliably storing large amounts of data. Although these codes provide optimal storage efficiency, they require significantly high network and disk usage during recovery of missing data. In this paper, we first present a study on the impact of recovery operations of erasure-coded ...
متن کاملClouddmss: Cloud-based Distributed Multimedia Streaming Service System for Heterogeneous Devices
With the recent appearance of various heterogeneous smart devices and the expansion of social network services, services that support the production and sharing of social media data are actively provided. The stable provision of such services requires a transcoding technology supporting the N-Screen service and distributed streaming technologies for providing a quality of service (QoS)-based ri...
متن کاملDesign and Implementation of a Cloud-based Distributed Multimedia Streaming Service (clouddmss) System
With the recent appearance of various heterogeneous smart devices and the expansion of social network services, services that support the production and sharing of social media data are actively provided. The stable provision of such services requires a transcoding technology supporting the N-Screen service and distributed streaming technologies for providing a quality of service (QoS)-based ri...
متن کامل